Gesture Recognition for Remote Collaborative Physical Tasks Using Tablet PCs

In this paper, we present effective and efficient gesture recognition algorithms for building a system to support remote collaborative physical tasks using tablet PCs. We discuss problems of gesture recognition in detail. We use a variable window to extract curvature changes to form invariant local features. We then employ a hierarchical classifier that consists of hidden Markov model (HMM) and decision tree classifiers. HMMs are utilized to classify closed gestures, while decision tree is used to classify open gestures. Experiment results show that the overall accuracy for all gestures is 96.4%. Accuracies for closed gestures and open gestures are 96.9% and 96.1% respectively. We have developed a prototype system integrating gesture and live video to support collaboration on physical tasks. Besides normal gesture recognition, the system also supports gesture fitting, freehand drawing, and the combination of the two. The system can support both human to computer interaction and human to human communication.